In the classical von Neumann computing architecture, computation and data storage is performed by separate modules, the central processing unit and the random access memory, respectively (cf., e.g., A. W. Burks, H. H. Goldstine, and J. von Neumann. Preliminary discussion of the logical design of an electronic computing instrument. Report 1946, U.S. Army Ordonance Department, 1946). A memory address sent to the random access memory gives access to the data content of one particular storage location. Associative memory structures are computing architectures in which computation and data storage is not separated (cf. T. Kohonen. Associative memory: a system theoretic approach. Springer, Berlin, 1977). For example, an associative memory can store a set of associations between pairs of (binary) patterns {(uμ→vμ): μ=1, . . . , M} (see FIGS. 1a and 1b).
FIG. 1a illustrates the memory tasks. In the storage phase, M associations of memory address patterns uμ and content patterns vμ are stored in the associative memory device (AM) as shown exemplarily in FIG. 1a (learning of associations between M pattern pairs, uμ→vμ). In the retrieval phase, the AM is addressed with an input query pattern ũ typically resembling one of the previously stored memory address patterns uμ1. The AM returns the retrieval result {circumflex over (v)} that should be similar to the associated memory content pattern vμ1, as shown exemplarily in FIG. 1b (retrieving phase).
Similar as in random access memory, a query pattern uμ entered in the associative memory can serve as address for accessing the associated pattern vμ. However, the tasks performed by the two types of memory differ fundamentally. Random access is only defined for query patterns that are valid addresses, that is, for the set of u patterns used during storage. The random access task consists of returning the data record at the addressed location (look-up).
In contrast, associative memory structures accept arbitrary input query patterns ũ and the computation of any particular output involves all stored data records rather than a single one. Specifically, the associative memory task consists of comparing an input query pattern ũ with all stored addresses and returning an output pattern equal (or similar) to the pattern vμ associated with the memory address pattern uμ most similar to the input query pattern. Thus, the associative memory task includes the random access task but is not restricted to it. It also includes computations such as pattern completion, denoising or data retrieval using incomplete cues.
Neural associative memory structures are parallel implementations of associative memory in a network of neurons in which associations are stored in a set of synaptic weights W between neurons typically employing fast Hebbian-type learning methods (cf., e.g., J. Hertz, A. Krogh, and R. G. Palmer. Introduction to the theory of neural computation. Addison-Wesley, Redwood City, 1991). Associative networks are closely related to Hebbian cell assemblies (cf., e.g., G. Palm. Neural Assemblies. An Alternative Approach to Artificial Intelligence. Springer, Berlin, 1982) and play an important role in neuroscience as models of neural computation for various brain structures, in particular neocortex, hippocampus, and cerebellum.